Mildly Penalized Maximum Likelihood Estimation of Genetic Covariances Matrices without Tuning

INTRODUCTION Estimates of genetic covariance matrices, ΣG, are known to be afflicted by substantial sampling errors, increasing markedly with the number of traits considered. ‘Regularization’, i.e. modification of estimators to reduce sampling variation at the expense of a small, additional bias, has been advocated to obtain estimates closer to the population values. An early suggestion by Hayes and Hill (1981) has been to shrink the canonical eigenvalues, λi, i.e. the eigenvalues of Σ−1 P ΣG (with ΣP the phenotypic covariance matrix), towards their mean. As shown by Meyer and Kirkpatrick (2010), the analogue in a maximum likelihood framework is to maximize the likelihood subject to a penalty proportional to the variance among the estimates of λi. Neither authors provided guidelines on how to determine the amount of shrinkage to be applied. While cross-validation techniques allow estimation of so-called ‘tuning factors’, this proved laborious and only moderately successful (Meyer 2011). An simple alternative is to apply a mild, default penalty which, while not providing maximum benefits, will yield stable estimates and worthwhile reductions in ‘loss’, i.e. the average deviations of estimates from population values. This is similar to the concept of weakly informative priors, which is gaining popularity in Bayesian estimation (e.g. Gelman 2006). This paper demonstrates the reductions in loss achievable using a default penalty on canonical eigenvalues.